Render a video on a Cube #322 #862

robinroy03 · 2024-02-20T04:28:25Z

Render a video on a cube (#322)

This PR adds a new actor - texture_on_cube to fury/actor.py which takes 6 different texture arguments for the different sides of a cube. It also adds the tutorial and tests for the same.

How it is done

I took 6 planes and merged them to form a cube.

How is video rendered

Video is rendered by changing the texture repeatedly.

Simple static texture demo:

from fury import actor, window
import numpy as np

arr_0 = np.full((720, 1280), 0, dtype=np.uint8)
arr_255 = np.full((720, 1280), 255, dtype=np.uint8)

arr_red = np.dstack((arr_255, arr_0, arr_0))
arr_white = np.full((720, 1280, 3), 255, dtype=np.uint8)
arr_aqua = np.dstack((arr_0, arr_255, arr_255))
arr_green = np.dstack((arr_0, arr_255, arr_0))
arr_blue = np.dstack((arr_0, arr_0, arr_255))
arr_yellow = np.dstack((arr_255, arr_255, arr_0))

cube = actor.texture_on_cube(arr_white, arr_red, arr_green, arr_blue, arr_yellow, arr_aqua)
cube2 = actor.texture_on_cube(arr_white, arr_red, arr_green, arr_blue, arr_yellow, arr_aqua, (3, 3, 3))

scene = window.Scene()
scene.add(*cube)
scene.add(*cube2)

window.show(scene, size=(1280, 720))

FURY.0.1.0.dev5378+gea0fb16.2024-03-17.15-17-32.mp4

pep8speaks · 2024-02-20T04:28:31Z

Hello @robinroy03, Thank you for updating!

Cheers! There are no style issues detected in this Pull Request. 🍻 To test for issues locally, pip install flake8 and then run flake8 fury.

Comment last updated at 2024-05-01 10:27:17 UTC

skoudoro · 2024-02-20T14:40:20Z

Hi @robinroy03,

Thank you for this, I will try tomorrow and give you an update

skoudoro · 2024-02-20T14:40:41Z

during this time, can you fix all the pep8 issues?

…vements, fixed the bug when video ends

robinroy03 · 2024-02-20T16:39:22Z

@skoudoro I've fixed all the pep8 issues and have placed some of my thoughts in the code as comments. (and also in the PR description)
I'll add tests and finish this once you're ok with them.

skoudoro

Hi @robinroy03,

Thank you for this. This is a good start.

The goal was to show, a different video at each face of the cube. if you want to use the same video, they should start at different time point.

it is ok to use the texture, but you need to control the different part of your texture.

After, I will be able to look deeper at the code.

Thank you in advance for your future update.

robinroy03 · 2024-02-25T08:13:13Z

Hi @skoudoro,

Are there any inbuilt VTK functions that do this? (mapping different textures to different sides of a cube)
Because when I searched I found this (https://discourse.vtk.org/t/how-to-apply-different-textures-to-each-face-of-a-tetrahedron/2333/2), and I'm not sure if I should go this path directly.

Are there any resources for UV-Mapping textures in VTK? (I referred both VTKUsersGuide and VTKTextbook, found no examples)

robinroy03 · 2024-02-25T15:11:33Z

I found this fury/io/load_cubemap_texture function and used it. But I am unable to map the cubemap to the cube. (the below code does not work).

import vtk

from fury import actor, window
from fury.io import load_cubemap_texture


tx = load_cubemap_texture(fnames = [
    "negx.jpg",
    "negy.jpg",
    "negz.jpg",
    "posx.jpg",
    "posy.jpg",
    "posz.jpg"
])

scene = window.Scene()

cubeSource = vtk.vtkCubeSource()

mapper = vtk.vtkPolyDataMapper()
mapper.SetInputConnection(cubeSource.GetOutputPort())

cube_actor = vtk.vtkActor()
cube_actor.SetTexture(tx)
cube_actor.SetMapper(mapper)

scene.add(cube_actor)
window.show(scene)

Can I get some help regarding the general direction?

Or should I take 6 PlaneSources and merge them?

robinroy03 · 2024-03-03T10:26:03Z

@skoudoro,

Please check the new commit, I've used 6 planes to make the cube. I can use vtkAssembly() to make them 1 single actor if needed.

Use a cubemap texture from here to test.

If this is ok, I'll finish this code with some more methods such as texture_update, get_actor etc ... (or should I format the code differently? I believe this class should go inside actor.py)

Demo Video:

FURY.0.9.0.2024-03-03.15-53-38.mp4

This code is not rendering a video, but it'll be done in texture_update. I just wanted to know whether my general direction is correct & feedback about how I'm organizing the code.

codecov · 2024-03-03T11:37:21Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.46%. Comparing base (14851c6) to head (b26842a).

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #862      +/-   ##
==========================================
+ Coverage   84.43%   84.46%   +0.03%     
==========================================
  Files          44       44              
  Lines       10537    10559      +22     
  Branches     1423     1432       +9     
==========================================
+ Hits         8897     8919      +22     
  Misses       1266     1266              
  Partials      374      374

Files	Coverage Δ
fury/actor.py	`85.85% <100.00%> (+0.25%)`	⬆️
fury/lib.py	`100.00% <100.00%> (ø)`

…use it is redundant when get_actor() alreadt exists

robinroy03 · 2024-03-12T18:12:41Z

@skoudoro
I've finished everything. Please take a look and let me know if I've to make modifications.

…mplify code, modified the tests for the coordinate change

skoudoro

Hi @robinroy03,

Thank you for this. See my initial comment. My main concern is about the use of a class. I think everything can be simpler and easier to maintain with function

docs/examples/viz_play_cube.py

fury/actor.py

…ade the tutorial more descriptive, and included the changes skoudoro requested during review

robinroy03 · 2024-03-19T19:32:15Z

I've made the changes. For test_actors, I only did a test for 1 side colour render. I avoided remaining because it all comes from the same pipeline, so must be similar.

skoudoro

Hi @robinroy03,

After skimming the code , it looks much better! Please, check our other tutorials to see how it is formatted. Currently, your tutorial will not render well. You can try to render it locally.

I will try to look deeper on your code later today. Thanks for this

robinroy03 · 2024-03-21T13:14:14Z

@skoudoro, I'm sorry but how do I render it locally? Every time I try I get a segf (it's an open issue #481).

I did create docs (by adding the disable examples flag), but my new code is not shown there. What is the procedure to add my code to the docs?

skoudoro · 2024-03-21T13:43:20Z

What is the procedure to add my code to the docs?

you need to add it in docs/examples/_valid_examples.toml

Every time I try I get a segf

for now, run it multiple times consecutively

…on_cube. Updated _valid_examples for doc generation

robinroy03 · 2024-03-21T17:41:22Z

All the said modifications are done. I also committed the new file _valid_examples.toml.

I think this is a bug, but viz_play_cube.png will be all black unless I turn interactive = True. It takes a screenshot immediately, even before the actors are rendered. But if we give it some time after turning interactivity on, it works fine (since the actors are rendered).
I also noticed these issues with window.snapshot where they take screenshots very immediately even before some actors are rendered.

…ed inwards. This might affect if we introduce some optimizations), also some minor QOL changes. Deleted vtkprop because it was unused, it was introduced previously but now it is not needed

robinroy03 · 2024-04-29T20:09:04Z

Hi @skoudoro, PTAL.

Thank you.

docs/examples/viz_play_cube.py

fury/actor.py

skoudoro

Hi @robinroy03,

Thank you for this.

6 actors is a lot. I am not sure we would like to have this in actor.py. Maybe the function will be only in the tutorial. I will need the opinion of other members of the team.
See above some comments

…tuple optional, uint8 constraint removal, note about 6 actors

robinroy03 · 2024-05-01T10:50:23Z

@skoudoro,

All the requested changes are made. Should I modify the function to be placed inside the tutorial? We'd discussed using 6 different actors to render the cube multiple times.

skoudoro · 2024-05-01T13:09:55Z

All the requested changes are made.

Thanks!

Should I modify the function to be placed inside the tutorial?

I do not know yet

We'd discussed using 6 different actors to render the cube multiple times.

I know... the fact that we talk about it multiple time show that I am not completely satisfy with this solution because I can see long term issue with it. I have hard time to block time to dig in the texture actor to see if there are alternatives or something that we are missing.

robinroy03 · 2024-05-01T13:42:55Z

ok, I'll also keep researching whether there are better ways to do this.

video on a cube

deb99c6

robinroy03 added 3 commits February 20, 2024 21:40

import vtk through fury lib.py, added some comments on possible impro…

41848f7

…vements, fixed the bug when video ends

fixing pep8 one line at a time

2773729

fixing pep8 one line at a time (part 2)

6feeeb0

robinroy03 mentioned this pull request Feb 22, 2024

Add glow effects #326

Open

skoudoro requested changes Feb 23, 2024

View reviewed changes

robinroy03 and others added 4 commits March 2, 2024 12:43

Merge branch 'fury-gl:master' into cube_viz

d4b3344

modified cube viz with different textures on all 6 sides

f58f41d

pep8 fix

ed688ca

pep8

1887ccb

robinroy03 requested a review from skoudoro March 3, 2024 10:26

robinroy03 added 6 commits March 10, 2024 21:36

textured cube final version without tests

5a7c6fc

tutorial for viz_play_cube

6fc3dcd

pep8 fix

a351499

pep8

78f8135

modified to elininate circular import error, removed get_scene() beca…

724ee56

…use it is redundant when get_actor() alreadt exists

added tests and fixed some docstrings, also added to lib.py

6a84176

using isintance for the type comparison

f735ad9

robinroy03 force-pushed the cube_viz branch from e66b6c0 to f735ad9 Compare March 15, 2024 19:30

Added center coordinate parameter, used util.py helper function to si…

c141f31

…mplify code, modified the tests for the coordinate change

skoudoro reviewed Mar 18, 2024

View reviewed changes

docs/examples/viz_play_cube.py Outdated Show resolved Hide resolved

docs/examples/viz_play_cube.py Show resolved Hide resolved

docs/examples/viz_play_cube.py Outdated Show resolved Hide resolved

fury/actor.py Outdated Show resolved Hide resolved

Removed classes, made it functional. Changed the tests accordingly. M…

1953851

…ade the tutorial more descriptive, and included the changes skoudoro requested during review

robinroy03 requested a review from skoudoro March 19, 2024 19:34

robinroy03 and others added 2 commits March 20, 2024 14:51

Merge branch 'fury-gl:master' into cube_viz

0367526

fixed a coordinate bug, now it is perfectly aligned to the center

dc74532

skoudoro reviewed Mar 21, 2024

View reviewed changes

robinroy03 added 2 commits March 21, 2024 23:01

fixed docs, made it cleaner. Fixed uneven variable names for texture_…

264721b

…on_cube. Updated _valid_examples for doc generation

small typo fix

20d668d

robinroy03 requested a review from skoudoro March 21, 2024 17:41

fixed the normal directions (it now points outwards, earlier it point…

62ac13c

…ed inwards. This might affect if we introduce some optimizations), also some minor QOL changes. Deleted vtkprop because it was unused, it was introduced previously but now it is not needed